大部分强化学习理论都建立在计算上难以实施的甲板上。专门用于在部分可观察到的马尔可夫决策过程（POMDP）中学习近乎最佳的政策，现有算法要么需要对模型动态（例如确定性过渡）做出强有力的假设，要么假设访问甲骨文作为解决艰难的计划或估算问题的访问子例程。在这项工作中，我们在合理的假设下开发了第一个用于POMDP的无Oracle学习算法。具体而言，我们给出了一种用于在“可观察” pomdps中学习的准化性时间端到端算法，其中可观察性是一个假设，即对国家而言，分离良好的分布诱导了分离良好的分布分布而不是观察。我们的技术规定了在不确定性下使用乐观原则来促进探索的更传统的方法，而是在构建策略涵盖的情况下提供了一种新颖的barycentric跨度应用。

部分可观察到的马尔可夫决策过程（POMDPS）是加强学习的自然和一般模型，以考虑到代理人对其当前国家的不确定性。在POMDPS的文献中，习惯性地假设在已知参数时计算最佳策略的规划Oracle，即使已知问题是计算的。几乎所有现有的规划算法都在指数时间内运行，缺乏可证明的性能保证，或者需要在每个可能的政策下对转换动态进行强烈的假设。在这项工作中，我们重新审视了规划问题并问：是否有自然和积极的假设，使计划变得容易？我们的主要结果是用于规划（一步）可观察POMDPS的QuasioInomial-time算法。具体而言，我们假设各国的分离良好的分布导致分开的观察分布，因此观察结果在每一步中至少有一些信息。至关重要的是，这个假设没有对POMDP的过渡动态的限制;尽管如此，它意味着近乎最佳的政策承认准简洁的描述，这通常不是真实的（在标准的硬度假设下）。我们的分析基于滤波器稳定性的新定量界限 - 即潜在状态的最佳滤波器的速率忘记其初始化。此外，在指数时间假设下，我们证明了在可观察POMDPS中规划的匹配硬度。

给定真实的假设类$ \ mathcal {h} $，我们在什么条件下调查有一个差异的私有算法，它从$ \ mathcal {h} $给出的最佳假设.I.i.d.数据。灵感来自最近的成果的二进制分类的相关环境（Alon等，2019; Bun等，2020），其中显示了二进制类的在线学习是必要的，并且足以追随其私人学习，Jung等人。 （2020）显示，在回归的设置中，$ \ mathcal {h} $的在线学习是私人可读性所必需的。这里的在线学习$ \ mathcal {h} $的特点是其$ \ eta $-sequentient胖胖子的优势，$ {\ rm sfat} _ \ eta（\ mathcal {h}）$，适用于所有$ \ eta> 0 $。就足够的私人学习条件而言，Jung等人。 （2020）显示$ \ mathcal {h} $私下学习，如果$ \ lim _ {\ eta \ downarrow 0} {\ rm sfat} _ \ eta（\ mathcal {h}）$是有限的，这是一个相当限制的健康）状况。我们展示了在轻松的条件下，\ LIM \ INF _ {\ eta \ downarrow 0} \ eta \ cdot {\ rm sfat} _ \ eta（\ mathcal {h}）= 0 $，$ \ mathcal {h} $私人学习，为\ \ rm sfat} _ \ eta（\ mathcal {h}）$ \ eta \ dockarrow 0 $ divering建立第一个非参数私人学习保证。我们的技术涉及一种新颖的过滤过程，以输出非参数函数类的稳定假设。

我们研究了非参数在线回归中的快速收敛速度，即遗憾的是关于具有有界复杂度的任意函数类来定义后悔。我们的贡献是两倍： - 在绝对损失中的非参数网上回归的可实现设置中，我们提出了一种随机适当的学习算法，该算法在假设类的顺序脂肪破碎尺寸方面获得了近乎最佳的错误。在与一类Littlestone维度$ D $的在线分类中，我们的绑定减少到$ d \ cdot {\ rm poly} \ log t $。这结果回答了一个问题，以及适当的学习者是否可以实现近乎最佳错误的界限;以前，即使在线分类，绑定的最知名错误也是$ \ tilde o（\ sqrt {dt}）$。此外，对于真实值（回归）设置，在这项工作之前，界定的最佳错误甚至没有以不正当的学习者所知。 - 使用上述结果，我们展示了Littlestone维度$ D $的一般总和二进制游戏的独立学习算法，每个玩家达到后悔$ \ tilde o（d ^ {3/4} \ cdot t ^ {1 / 4}）$。该结果概括了Syrgkanis等人的类似结果。 （2015）谁表明，在有限的游戏中，最佳遗憾可以从普通的o（\ sqrt {t}）$中的$ o（\ sqrt {t}）为游戏设置中的$ o（t ^ {1/4}）$。要建立上述结果，我们介绍了几种新技术，包括：分层聚合规则，以实现对实际类别的最佳错误，Hanneke等人的适当在线可实现学习者的多尺度扩展。 （2021），一种方法来表明这种非参数学习算法的输出是稳定的，并且证明Minimax定理在所有在线学习游戏中保持。

最近，Daskalakis，Fisselson和Golowich（DFG）（Neurips`21）表明，如果所有代理在多人普通和正常形式游戏中采用乐观的乘法权重更新（OMWU），每个玩家的外部遗憾是$ o（\ textrm {polylog}（t））$ the游戏的$重复。我们从外部遗憾扩展到内部遗憾并交换后悔，从而建立了以$ \ tilde {o}的速率收敛到近似相关均衡的近似相关均衡（t ^ { - 1}）$。由于陈和彭（神经潜行群岛20），这实质上提高了以陈和彭（NEURIPS20）的相关均衡的相关均衡率，并且在无遗憾的框架内是最佳的 - 以$ $ $ to to polylogarithmic因素。为了获得这些结果，我们开发了用于建立涉及固定点操作的学习动态的高阶平滑的新技术。具体而言，我们确定STOLTZ和LUGOSI（Mach Learn`05）的无内部遗憾学习动态在组合空间上的无外部后悔动态等效地模拟。这使我们可以在指数大小的集合上交易多项式大型马尔可夫链的计算，用于在指数大小的集合上的（更良好的良好）的线性变换，使我们能够利用类似的技术作为DGF到接近最佳地结合内心遗憾。此外，我们建立了$ O（\ textrm {polylog}（t））$ no-swap-recreet遗憾的blum和mansour（bm）的经典算法（JMLR`07）。我们这样做是通过基于Cauchy积分的技术来介绍DFG的更有限的组合争论。除了对BM的近乎最优遗憾保证的阐明外，我们的论点还提供了进入各种方式的洞察，其中可以在分析更多涉及的学习算法中延长和利用DFG的技术。

Effective management of public shared spaces such as car parking space, is one challenging transformational aspect for many cities, especially in the developing World. By leveraging sensing technologies, cloud computing, and Artificial Intelligence, Cities are increasingly being managed smartly. Smart Cities not only bring convenience to City dwellers, but also improve their quality of life as advocated for by United Nations in the 2030 Sustainable Development Goal on Sustainable Cities and Communities. Through integration of Internet of Things and Cloud Computing, this paper presents a successful proof-of-concept implementation of a framework for managing public car parking spaces. Reservation of parking slots is done through a cloud-hosted application, while access to and out of the parking slot is enabled through Radio Frequency Identification (RFID) technology which in real-time, accordingly triggers update of the parking slot availability in the cloud-hosted database. This framework could bring considerable convenience to City dwellers since motorists only have to drive to a parking space when sure of a vacant parking slot, an important stride towards realization of sustainable smart cities and communities.

Despite recent success in large language model (LLM) reasoning, LLMs still struggle with hierarchical multi-step reasoning like generating complex programs. In these cases, humans often start with a high-level algorithmic design and implement each part gradually. We introduce Parsel, a framework enabling automatic implementation and validation of complex algorithms with code LLMs, based on hierarchical function descriptions in natural language. Parsel can be used across domains requiring hierarchical reasoning, e.g. code synthesis, theorem proving, and robotic planning. We demonstrate Parsel's capabilities by using it to generate complex programs that cannot currently be automatically implemented from one description and backtranslating Python programs in the APPS dataset. Beyond modeling capabilities, Parsel allows problem-solving with high-level algorithmic designs, benefiting both students and professional programmers.

Current image generation models struggle to reliably produce well-formed visual text. In this paper, we investigate a key contributing factor: popular text-to-image models lack character-level input features, making it much harder to predict a word's visual makeup as a series of glyphs. To quantify the extent of this effect, we conduct a series of controlled experiments comparing character-aware vs. character-blind text encoders. In the text-only domain, we find that character-aware models provide large gains on a novel spelling task (WikiSpell). Transferring these learnings onto the visual domain, we train a suite of image generation models, and show that character-aware variants outperform their character-blind counterparts across a range of novel text rendering tasks (our DrawText benchmark). Our models set a much higher state-of-the-art on visual spelling, with 30+ point accuracy gains over competitors on rare words, despite training on far fewer examples.

Large "instruction-tuned" language models (finetuned to respond to instructions) have demonstrated a remarkable ability to generalize zero-shot to new tasks. Nevertheless, they depend heavily on human-written instruction data that is limited in quantity, diversity, and creativity, therefore hindering the generality of the tuned model. We introduce Self-Instruct, a framework for improving the instruction-following capabilities of pretrained language models by bootstrapping off its own generations. Our pipeline generates instruction, input, and output samples from a language model, then prunes them before using them to finetune the original model. Applying our method to vanilla GPT3, we demonstrate a 33% absolute improvement over the original model on Super-NaturalInstructions, on par with the performance of InstructGPT_001, which is trained with private user data and human annotations. For further evaluation, we curate a set of expert-written instructions for novel tasks, and show through human evaluation that tuning GPT3 with Self-Instruct outperforms using existing public instruction datasets by a large margin, leaving only a 5% absolute gap behind InstructGPT_001. Self-Instruct provides an almost annotation-free method for aligning pre-trained language models with instructions, and we release our large synthetic dataset to facilitate future studies on instruction tuning.

Language models have recently achieved strong performance across a wide range of NLP benchmarks. However, unlike benchmarks, real world tasks are often poorly specified, and agents must deduce the user's intended behavior from a combination of context, instructions, and examples. We investigate how both humans and models behave in the face of such task ambiguity by proposing AmbiBench, a new benchmark of six ambiguously-specified classification tasks. We evaluate humans and models on AmbiBench by seeing how well they identify the intended task using 1) instructions with varying degrees of ambiguity, and 2) different numbers of labeled examples. We find that the combination of model scaling (to 175B parameters) and training with human feedback data enables models to approach or exceed the accuracy of human participants across tasks, but that either one alone is not sufficient. In addition, we show how to dramatically improve the accuracy of language models trained without large-scale human feedback training by finetuning on a small number of ambiguous in-context examples, providing a promising direction for teaching models to generalize well in the face of ambiguity.